Introduce opentelemetry in rewatch and setup new test infrastructure#8241
Open
nojaf wants to merge 16 commits intorescript-lang:masterfrom
Open
Introduce opentelemetry in rewatch and setup new test infrastructure#8241nojaf wants to merge 16 commits intorescript-lang:masterfrom
nojaf wants to merge 16 commits intorescript-lang:masterfrom
Conversation
rescript
@rescript/darwin-arm64
@rescript/darwin-x64
@rescript/linux-arm64
@rescript/linux-x64
@rescript/runtime
@rescript/win32-x64
commit: |
6fc10da to
0ce96cc
Compare
Add optional OTLP tracing export to rewatch, controlled by the OTEL_EXPORTER_OTLP_ENDPOINT environment variable. When set, rewatch exports spans via HTTP OTLP; when unset, tracing is a no-op. Instrument key build system functions (initialize_build, incremental_build, compile, parse, clean, format, packages) with tracing spans and attributes such as module counts and package names. Restructure main.rs to support telemetry lifecycle (init/flush/shutdown) and fix show_progress to use >= LevelFilter::Info so -v/-vv don't suppress progress messages. Also print 'Finished compilation' in plain_output mode during watch full rebuilds. Introduce a new Vitest-based test infrastructure in tests/rewatch_tests/ that replaces the bash integration tests. Tests spawn rewatch with an OTLP endpoint pointing to an in-process HTTP receiver, collect spans, and snapshot the resulting span tree for deterministic assertions. Update CI, Makefile, and scripts/test.js to use the new test runner.
When stdin is a pipe (not a TTY), spawn a background thread that monitors for EOF. This allows a parent process (such as the test harness) to signal a graceful shutdown by closing stdin, without relying on signals or lock file removal.
Add mtime and content-hash based deduplication to filter out phantom and duplicate file system events. Normalize event kinds from atomic writes (temp file + rename) so they are treated as content modifications rather than create/remove cycles that trigger unnecessary full rebuilds. This fixes issues on macOS (Create events from atomic writes), Linux (duplicate inotify IN_MODIFY events), and Windows (Remove+Rename sequences from atomic writes).
On Windows, bsc writes CRLF to stdout in text mode. When the original source file uses LF line endings, the formatted output would introduce unwanted CRLF conversions. Detect the original file's line ending style and normalize the formatted output to match.
0ce96cc to
9ba38d7
Compare
Member
Author
|
@rolandpeelen and @jfrolich this is already a good first carve out worth reviewing. Changes to rewatch are minimal but I need some small fixes for the new test suite. Next steps would be to port more tests, but you can already take a look at the rewatch changes. |
Propagate parent span through rayon in build.parse so build.parse_file spans are properly nested under build.parse instead of appearing as orphaned root spans. Enrich build.compile_file span with package, suffix, module_system, and namespace attributes for better observability. Handle invalid config changes gracefully during watch mode: replace .expect() with match to report the error and continue watching, allowing the user to fix the config without restarting the watcher.
Add 7 new fixture packages to cover more configuration dimensions: - commonjs: CommonJS module output with .bs.js suffix - namespaced: namespace package with TestNS - noop-ppx: lightweight cross-platform no-op PPX for testing - with-deps: package depending on rescript-bun for clean tests - with-dev-deps: multi-source dirs with dev dependencies - with-jsx: JSX v4 with @rescript/react - with-ppx: PPX integration using noop-ppx Enhance test helpers: - Normalize CRLF line endings in process output for Windows - Support .bs.js artifacts in sandbox cleanup detection - Add createCli, readFileInSandbox, writeFileInSandbox helpers - Add OTEL config for build.parse_file and enriched compile_file spans - Exclude noop-ppx from biome linting (CommonJS required)
Add tests for core build functionality: - Build from a package subdirectory - No stale artifacts on second build - Filter flag to compile only matching modules Add build error tests: - Parse error reporting with file location - Type error reporting - Errors when a dependency module is deleted - Circular dependency detection
Add module operation tests: - File rename with and without dependents - Duplicate module name detection - Interface file compilation and error cases Add namespace package tests: - Build with namespace flag - Namespace in compiler args - File rename in namespaced package Add dev-dependency tests: - Dev source compiles with dev dependencies - Non-dev source cannot use dev dependencies - Clean removes dev source artifacts
Add build config tests: - Experimental feature flags (valid, invalid key, invalid format) - After-build hook execution (success and failure) - Warning configuration in compiler args - Warn-error CLI override - Deprecated and unknown config field warnings Add module system tests: - CommonJS package with .bs.js suffix - CommonJS in compiler args - Suffix change triggers rebuild - Duplicate package-spec suffix error Add PPX integration tests using lightweight noop-ppx: - PPX build produces output - PPX flags in parser args - PPX flags not in compiler args Add JSX tests: - JSX v4 build with @rescript/react - JSX flags in parser args - JSX preserve flag
Add tests for scoped clean, node_modules dependency cleanup, and verifying no false compiler-update message after clean+rebuild.
Add format tests: - Stdin formatting for .res and .resi - Single file and all-files formatting - Subdirectory-scoped formatting - Check mode (pass and fail cases) Add compiler-args tests: - CWD invariance (same output from root and subdirectory) - Warning flags in both parser and compiler args
Verify that a concurrent build is prevented while watch mode holds the lock file.
Add watch mode tests: - New file creation triggers compilation - Warning persistence across incremental builds - Config change triggers full rebuild - Changes outside source dirs are ignored - Missing source folder does not crash watcher - Invalid config change recovery (watcher keeps running) - File rename removes old artifacts and compiles new file - File deletion removes artifacts
Tracing spans are thread-local, so compile_file spans created inside Rayon's par_iter had no parent connection to the compile_wave span on the main thread. Pass the wave span explicitly via `parent: &wave_span` to establish the correct parent-child relationship.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a subset of #8231.
I wish to introduce open telemetry in Rewatch and migrate the bash tests to a new test suite.
The new tests would look like:
Where we capture all telemetry traces produced inside
runRewatchTestand extract a deterministic snapshot of this:With this telemetry we can easily update snapshots and assert all still works as expected.
By using Vitest we also have a proper test runner, so can run individual tests.
All tests also run in a sandbox folder in the
/tmp, we they cannot interfere with each other.And they can run in parallel.
The goal is too replace and remove the existing bash tests.